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1  Brief  Description  of  Results 


Below  we  briefly  describe  some  specific  areas  of  research  that  have  been  pursued  and  the  results 
obtained  during  the  course  of  the  project  (namely  8/1/92  to  7/31/95).  In  the  next  section,  a  list 
publications  resulting  from  this  award  is  provided. 

1.1  Class  Identification  and  Discernibility 

We  have  been  investigating  a  general  problem  for  class  identification  that  consists  of  trying  to  iden¬ 
tify  to  which  of  two  classes  or  Ai  an  unknown  object  or  distribution  belongs  given  a  information 
presented  sequentially.  We  have  considered  a  variety  of  success  criteria  and  have  characterized  when 
class  identification  can  be  performed.  We  have  also  considered  applications  to  discrete  geometry, 
system  identification,  and  language  identification.  A  paper  on  these  results  appeared  in  Annals  of 
Statistics  [6]  and  another  paper  appeared  in  IEEE  Transaction  on  Information  Theory  [10].  We 
also  investigated  a  problem  which  is  a  hybrid  of  standard  PAG  learning  and  the  class  identification 
problem.  These  results  take  steps  towards  establishing  connections  between  the  two  areas.  This 
work  was  presented  at  the  6th  Annual  Workshop  on  Computational  Learning  Theory  [35].  We 
subsequently  generalized  these  results  as  well  as  extended  and  provided  a  new  proof  technique  for 
results  of  others.  This  work  was  presented  at  the  1995  Int.  Symp.  on  Information  Theory. 

1.2  Learning  Under  a  Family  of  Probability  Measures 

We  have  been  investigating  PAG  learning  with  respect  to  a  class  of  distributions.  Near  the  start 
of  this  project,  we  resolved  a  conjecture  regarding  this  problem  that  had  been  open  for  about  five 
years.  A  paper  on  this  work  appeared  in  IEEE  Transactions  on  Information  Theory  in  May  ’94 
[11].  During  this  period  we  also  obtained  some  results  which  begin  to  characterize  learnability 
under  general  classes  of  probability  measures.  This  work  was  presented  at  the  1994  International 
Symposium  on  Information  Theory  [24]  and  also  appeared  in  IEEE  Transactions  on  Information 
Theory  [7]. 

1.3  Regression  Estimation  for  Arbitrary  Processes 

Our  work  in  this  area  began  by  exploring  on-line  learning  of  functions  in  which  predictions  are  made 
sequentially  as  data  arrives  and  a  cumulative  success  criterion  is  used.  Results  from  this  work  were 
presented  at  the  1993  American  Control  Conference  [33]  and  at  the  6th  Annual  Workshop  on 
Computational  Learning  Theory  [34].  We  then  extended  the  analysis  to  a  more  general  statistical 
framework  with  the  aim  of  obtaining  rates  of  convergence  for  nearest-neighbor  estimation  under 
arbitrary  sampling.  Some  of  these  results  were  presented  at  the  1994  International  Symposium  on 
Information  Theory  [23]  and  the  1994  Information  Theory  and  Statistics  Workshop  [27].  A  paper 
on  this  work  appeared  in  IEEE  Transactions  on  Information  Theory  [5].  We  then  further  extended 
the  work  to  deal  with  a  broad  class  of  non-parametric  estimators.  We  introduced  a  new  notion  of 
Cesaro  consistency  for  which  the  time-average  of  the  risk  must  tend  to  zero,  and  have  shown  that 
Cesaro  consistency  can  be  achieved  under  mild  conditions  for  arbitrary  processes.  These  results 
were  presented  at  the  1995  International  Symposium  on  Information  Theory  [20].  More  recently,  we 
used  similar  techniques  for  universal  prediction  of  the  output  of  nonlinear  fading  memory  systems. 
These  results  were  presented  at  the  1995  Conference  on  Decision  and  Control  [15]  and  more  refined 
results  have  also  been  submitted  for  publication  in  IEEE  Transactions  on  Automatic  Control  [1]. 
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1.4  Stochastic  Approximation 


We  have  analyzed  the  classical  Robbins-Monro  and  Kiefer- Wolfowitz  stochastic  approximation  al¬ 
gorithms  under  arbitrary  disturbances.  Our  main  result  characterizes  convergence  in  terms  of  a 
condition  on  the  noise  sequence  which  we  call  persistently  disturbing,  that  turns  out  to  be  equiv¬ 
alent  to  the  classical  Kushner-Clark  condition.  Specifically,  we  show  that  the  algorithms  converge 
under  each  fixed  noise  sequence  iff  the  noise  is  not  persistently  disturbing.  Our  result  is  stronger 
than  previous  results  and  in  fact  we  can  obtain  a  number  of  previous  results  as  simple  corollaries. 
Furthermore,  our  proof  techniques  are  elementary.  Portions  of  this  work  were  presented  at  the 
32nd  IEEE  Conference  on  Decision  and  Control  [29],  the  1994  American  Control  Conference  [26], 
and  the  1994  SIAM  Annual  meeting  [25].  A  paper  on  this  work  appeared  in  IEEE  Transactions 
on  Automatic  Control  [8].  In  further  work,  we  extended  and  showed  the  equivalence  of  various 
conditions  for  the  convergence  of  the  Robbins-Monro  algorithm.  This  work  was  presented  at  the 
32nd  AUerton  Conference  [22]  and  appeared  in  Advances  in  Applied  Probability  [4]. 

1.5  Bandit  Problems 

The  two-armed  bandit  is  perhaps  the  simplest  problem  which  exhibits  the  weU-known  exploration 
vs.  exploitation  (or  identification  vs.  control)  tradeoff.  During  this  period  we  formulated  a  gener¬ 
alization  of  the  classical  two-armed  bandit  problem  which  allows  the  selection  of  which  to  play  to 
depend  on  a  side  observation.  Bounds  on  the  achievable  regret  for  this  problem  were  obtained  using 
results  from  PAC  learning.  Hence,  this  work  also  relates  recent  work  in  non-parametric  estimation 
to  classical  bandit  problems.  This  work  was  presented  at  the  31st  AUerton  Conference  [30]. 

1.6  Geometric  reconstruction 

In  previous  work,  we  considered  a  problem  in  geometric  reconstruction  (recovering  a  curve  from 
fine  crossing  data)  and  used  results  from  learning  to  obtain  bounds  on  the  data  required  for  re¬ 
construction  to  a  prescribed  accuracy.  During  this  project,  we  appUed  similar  techniques  to  the 
problem  of  convex  set  reconstruction  from  various  types  of  data.  This  work  was  presented  at  the 
1993  Conference  on  Information  Sciences  and  Systems  [32].  We  also  did  some  related  work  on  the 
consistency  of  support  plane  data  in  higher  dimensions.  This  appeared  in  Journal  of  Mathematical 
Imaging  and  Vision  [9]. 

1.7  Hybrid  Systems 

There  has  been  a  great  deal  of  recent  interest  in  hybrid  systems.  One  problem  We  studied  is  the 
reduction  of  timed  automata.  Previous  results  had  shown  that  such  systems  could  be  reduced  to 
a  single  finite  state  automaton  for  certain  simple  enabling  regions.  We  have  extended  these  results 
for  quite  general  enabling  regions.  This  work  was  presented  at  the  33rd  Conference  on  Decision 
and  Control  [28]  and  a  manuscript  has  been  submitted  for  publication  in  Discrete  Event  dynamic 
Systems  [2].  Another  problem  we  have  considered  is  the  design  of  controller  switching  policies 
based  on  output  prediction  errors.  We  characterized  memory  requirements  for  convergent  decision 
rules  and  showed  their  use  in  adaptive  control  based  on  switching  between  a  set  of  pre-selected 
controllers.  This  result  appeared  in  IEEE  Transactions  on  Automatic  Control  [3]. 
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2  Publications  Resulting  From  This  Award 


Journals 

[1]  S.R.  Kulkarni  and  S.E.  Posner,  “Nonparametric  Output  Prediction  for  Nonlinear  Fading  Mem¬ 
ory  Systems,”  submitted  to  IEEE  Transactions  on  Automatic  Control. 

[2]  S.  Di  Gennaro,  C.  Horn,  S.R.  Kulkarni,  P.J.  Ramadge,  “Reduction  of  Timed  Hybrid  Systems,” 
submitted  to  Journal  of  Discrete  Event  Dynamic  Systems. 

[3]  S.R.  Kulkarni  and  P.J.  Ramadge,  “Model  and  Controller  Switching  Policies  Based  on  Output 
Prediction  Errors,”  IEEE  Transactions  on  Automatic  Control,  Vol.  41,  No.  11,  pp.  1594-1604, 
November,  1996. 

[4]  L-J.  Wang,  E.K.P.  Chong,  and  S.R.  Kulkarni,  “Equivalent  Necessary  and  Sufficient  Conditions 
on  Noise  Sequences  for  Stochastic  Approximation  Algorithms,”  Advances  in  Applied  Probabil¬ 
ity,  1996. 

[5]  S.R.  Kulkarni  and  S.E,  Posner,  “Rates  of  Convergence  of  Nearest  Neighbor  Estimation  under 
Arbitrary  Sampling,”  IEEE  Transactions  on  Information  Theory,  Vol.  41,  No.  4,  pp.  1028- 
1039,  July,  1995. 

[6]  S.R.  Kulkarni  and  0.  Zeitouni,  “A  General  Classification  Rule  for  Probability  Measures,”  An¬ 
nals  of  Statistics,  Vol.  23,  No.  4,  pp.  1393-1407,  1995. 

[7]  S.R.  Kulkarni  and  M.  Vidyasagar,  “Learning  Decision  Rules  for  Pattern  Classification  Under 
a  Family  of  Probability  Measures,”  IEEE  Transactions  on  Information  Theory,  Vol.  43,  No. 
1,  pp,  154-166,  January,  1997. 

[8]  S.R.  Kulkarni  and  C.  Horn,  “An  Alternative  Proof  for  Convergence  of  Stochastic  Approxima¬ 
tion  Algorithms,”  IEEE  Transactions  on  Automatic  Control,  Vol.  41,  pp.  419-424,  Mar.,  1996. 

[9]  W.C.  Karl,  S.R.  Kulkarni,  G.C.  Verghese,  and  A.S.  Willsky, “Local  Tests  for  Consistency  of 
Support  Hyperplane  Data,”  Journal  of  Mathematical  Imaging  and  Vision,  Vol.  6,  pp.  249-267, 
June,  1996. 

[10]  S.R.  Kulkarni  and  D.N.C.  Tse,  “A  Paradigm  for  Class  Identification  Problems,”  IEEE  Trans¬ 
actions  on  Information  Theory,  Vol.  40,  No.  3,  pp.  696-705,  May,  1994. 
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[11]  R.M.  Dudley,  S.R.  Kulkarni,  T.J.  Richardson,  0.  Zeitouni,  “A  Metric  Entropy  Bound  is  Not 
Sufficient  for  Learnability,”  IEEE  Transactions  on  Information  Theory,  Vol.  40,  No.  3,  pp. 
883-885,  May,  1994. 


Conferences 

[12]  J.  Hocherman-Prommer,  S.R.  Kulkarni,  P.J.  Ramadge,  “Time  Varying  Switched  Control  Based 
on  Output  Prediction  Errors,”  Proc.  S4th  Conference  on  Decision  and  Control,  pp.  2316-2317, 
Dec.,  1995. 

[13]  S.R.  Kulkarni  and  P.J.  Ramadge,  “Prediction  Error  Based  Controller  Selection  Policies,”  Proc. 
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[15]  S.R.  Kulkarni  and  S.E.  Posner,  “Universal  Prediction  of  Nonlinear  Systems,”  Proc.  34th  Con¬ 
ference  on  Decision  and  Control,  pp.  4024-4029,  Dec.,  1995. 

[16]  S.R.  Kulkarni  and  C.S.  Horn,  “Necessary  and  Sufficient  Conditions  for  Convergence  of  Stochas¬ 
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